tooldata: refactor and add database intfc db73

Note: this branch is force pushed, rebased to master at 8901872 ================================================================ This commit refactors internal tool data handling to: 1) Update and consolidate functions in a new directory: src/emc/tooldata 2) Define an applications interface: src/emc/tooldata/tooldata.hh 3) Support a new interface to a user-provided tool database management program specified by [EMCIO]DB_PROGRAM 4) Use a new mmapped tooldata interface between EMCIO and TASK while maintaining number of allowed pockets specified by CANON_POCKETS_MAX=1001 5) Deprecate tooldata in nml messages between EMCIO and TASK (legacy nml message method remains available using configure --enable-toolnml until removal) 6) Minimize impact to current users of master branch and continue to pass the suite of runtests on all supported buildbot machines. 7) Clarify program variable names that use a sequential index (idx) for accessing internal tooldata. Note: some legacy variable names like: pocketPrepped, selected_pocket,current_pocket are still used in a few places because of their inclusion in fielded user remap scripts or the tests suite. These items may be renamed in the future. ================================================================ Change Summary: 1) new directory: src/emc/tooldata tooldata.hh ------------ api tooldata_common.cc ----- shared functions tooldata_mmap.cc ------- mmap tooldata tooldata_nml.cc -------- nml tooldata tooldata_db.cc --------- database interface functions tool_watch.cc ---------- test utility tool_mmap_read.cc ------ test utility for mmap 3) updated docs: docs/src/getting-started/updating-linuxcnc.txt docs/src/config/ini-config.txt docs/src/code/code-notes.txt docs/src/gcode/overview.txt docs/src/gcode/tool-compensation.txt docs/src/remap/remap.txt docs/man1/iocontrol*.1 docs/src/tooldatabase/tooldatabase.txt (new) 3) removed (consolidated into other files): src/emc/task/iotaskintf.cc src/emc/ini/initool.hh src/emc/ini/initool.cc src/emc/rs274ngc/tool_parse.h src/emc/rs274ngc/tool_parse.cc src/emc/toolstore/ 4) emccanon call: removed GET_EXTERNAL_POCKETS_MAX (superseded by CANON_POCKETS_MAX) 5) hal: params to pins: iocontrol.N.tool-prep-index 6) gui updates (axis gui only) Print message if both [EMCIO]TOOL_TABLE and DB_PROGRAM are specified. Allow specifying tooleditor for a DB_PROGRAM Issue message for db_program/tool_table ambiguities 7) simulation configs: configs/sim/axis/db_demo/db_ran.ini configs/sim/axis/db_demo/db_nonran.ini ================================================================ Notes: 1) Testing has been aided by the extensive set of tests exercised by the runtests program employed to ensure the integrity of buildbot created deb packages. However, numerous functions are NOT tested by the runtests suite. Examples: a) remapped tool handling b) user-interfaces (ui) support for tool changes c) iov2 (ioControl_v2.cc -- no maintainer, deprecated) 2) Use of the mmapped interface between EMCIO and TASK reduces the EMC_TOOL_STAT message size from 112224 bytes to 224 bytes (64bit architecture) 3) The complete toolTable[] data for all (1001) CANON_POCKETS_MAX is no longer available should someone implement a *remote* ui with TASK and EMCIO implemented on *separate* machines via networked (TCP) nml messages. A single tool table instance for the current tool in spindle is included in the EMC_TOOL_STAT message.
LinuxCNC · Jan 31, 2021 · 2dbb2f6 · 2dbb2f6
1 parent 8901872
commit 2dbb2f6
Show file tree

Hide file tree

Showing 87 changed files with 5,470 additions and 2,692 deletions.
diff --git a/configs/common/linuxcnc.nml b/configs/common/linuxcnc.nml
@@ -8,11 +8,11 @@
 # Top-level buffers to EMC
 B emcCommand            SHMEM   localhost       8192    0       0       1       16 1001 TCP=5005 xdr queue confirm_write serial
 B emcError              SHMEM   localhost       8192    0       0       3       16 1003 TCP=5005 xdr queue
-B emcStatus             SHMEM   localhost       170000  0       0       2       16 1002 TCP=5005 xdr
+B emcStatus             SHMEM   localhost      20480    0       0       2       16 1002 TCP=5005 xdr
 
 # These are for the IO controller, EMCIO
-B toolCmd               SHMEM   localhost       2048    0       0       4       16 1004 TCP=5005 xdr
-B toolSts               SHMEM   localhost       131072  0       0       5       16 1005 TCP=5005 xdr
+B toolCmd               SHMEM   localhost       1024    0       0       4       16 1004 TCP=5005 xdr
+B toolSts               SHMEM   localhost       1024    0       0       5       16 1005 TCP=5005 xdr
 
 # Processes
 # Name          Buffer          Type    Host            Ops     server? timeout master? cnum

diff --git a/configs/common/linuxcnc_big.nml b/configs/common/linuxcnc_big.nml
@@ -0,0 +1,39 @@
+
+# Note: emcsvr is the master for all NML channels, and therefore is the
+# first to start.
+
+# Buffers
+# Name                  Type    Host            size    neut?   (old)   buffer# MP ---
+
+# Top-level buffers to EMC
+B emcCommand            SHMEM   localhost       8192    0       0       1       16 1001 TCP=5005 xdr queue confirm_write serial
+B emcError              SHMEM   localhost       8192    0       0       3       16 1003 TCP=5005 xdr queue
+B emcStatus             SHMEM   localhost       170000  0       0       2       16 1002 TCP=5005 xdr
+
+# These are for the IO controller, EMCIO
+B toolCmd               SHMEM   localhost       2048    0       0       4       16 1004 TCP=5005 xdr
+B toolSts               SHMEM   localhost       131072  0       0       5       16 1005 TCP=5005 xdr
+
+# Processes
+# Name          Buffer          Type    Host            Ops     server? timeout master? cnum
+
+P emc           emcCommand      LOCAL   localhost       RW      0       1.0     0       0
+P emc           emcStatus       LOCAL   localhost       W       0       1.0     0       0
+P emc           emcError        LOCAL   localhost       W       0       1.0     0       0
+P emc           toolCmd         LOCAL   localhost       W       0       1.0     0       0
+P emc           toolSts         LOCAL   localhost       R       0       1.0     0       0
+
+P emcsvr        emcCommand      LOCAL   localhost       W       1       1.0     1       2
+P emcsvr        emcStatus       LOCAL   localhost       R       1       1.0     1       2
+P emcsvr        emcError        LOCAL   localhost       R       1       1.0     1       2
+P emcsvr        toolCmd         LOCAL   localhost       W       1       1.0     1       2
+P emcsvr        toolSts         LOCAL   localhost       R       1       1.0     1       2
+P emcsvr        default         LOCAL   localhost       RW      1       1.0     1       2
+
+P tool          emcError        LOCAL   localhost       W       0       1.0     0       3
+P tool          toolCmd         LOCAL   localhost       R       0       1.0     0       3
+P tool          toolSts         LOCAL   localhost       W       0       1.0     0       3
+
+P xemc          emcCommand      LOCAL   localhost       W       0       10.0    0       10
+P xemc          emcStatus       LOCAL   localhost       R       0       10.0    0       10
+P xemc          emcError        LOCAL   localhost       R       0       10.0    0       10
diff --git a/configs/common/tool.tbl b/configs/common/tool.tbl
@@ -1,4 +1,4 @@
 T1 P1 D0.125000 Z+0.511000 ;1/8 end mill
 T2 P2 D0.062500 Z+0.100000 ;1/16 end mill
 T3 P3 D0.201000 Z+1.273000 ;#7 tap drill
-T99999 P99999 Z+0.100000 ;big tool number
+T99999 P123     Z+0.100000 ;big tool number
diff --git a/configs/sim/axis/db_demo/README b/configs/sim/axis/db_demo/README
@@ -0,0 +1,19 @@
+db_ran.ini       random_toolchanger
+db_nonran.ini nonrandom_toolchanger
+
+Each config starts with:
+  tool    numbers 10--19
+  pockets numbers 11--20
+
+tool 10 in pocket 11
+tool 11 in pocket 12
+...
+
+
+To get more information (stdout):
+
+1) export DB_SHOW=
+   show 'g' (get) command
+
+2) export DB_DEBUG=
+   show more database info
diff --git a/configs/sim/axis/db_demo/base.inc b/configs/sim/axis/db_demo/base.inc
@@ -0,0 +1,52 @@
+#-----------------------------------------
+# include file for common stanzas
+
+[DISPLAY]
+OPEN_FILE=./base.ngc
+POSITION_OFFSET=RELATIVE
+POSITION_FEEDBACK=ACTUAL
+DISPLAY = axis
+MAX_LINEAR_VELOCITY = 1
+# placeholder for an editor for a db_program:
+TOOL_EDITOR = ./db_editor.tcl
+
+[TASK]
+TASK = milltask
+CYCLE_TIME = 0.001
+
+[EMCMOT]
+EMCMOT = motmod
+SERVO_PERIOD = 1000000
+
+[HAL]
+HALUI=halui
+HALFILE = LIB:basic_sim.tcl -no_use_hal_manualtoolchange
+
+[TRAJ]
+NO_FORCE_HOMING = 1
+
+COORDINATES = XYZ
+LINEAR_UNITS = inch
+ANGULAR_UNITS = degree
+
+[KINS]
+JOINTS = 3
+KINEMATICS = trivkins coordinates=XYZ
+
+[JOINT_0]
+TYPE = LINEAR
+MAX_VELOCITY=1e99
+MAX_ACCELERATION=1e99
+HOME_SEQUENCE=0
+
+[JOINT_1]
+TYPE = LINEAR
+MAX_VELOCITY=1e99
+MAX_ACCELERATION=1e99
+HOME_SEQUENCE=0
+
+[JOINT_2]
+TYPE = LINEAR
+MAX_VELOCITY=1e99
+MAX_ACCELERATION=1e99
+HOME_SEQUENCE=0
diff --git a/configs/sim/axis/db_demo/base.ngc b/configs/sim/axis/db_demo/base.ngc
@@ -0,0 +1,23 @@
+#<toolno> = 12
+; demo file using tool from database
+
+t#<toolno>    (debug,tool:#<toolno>: zoffset=#5403)
+g4p1   ;dwell
+
+m6     (debug,m6  zoffset=#5403)
+g4p1   ;dwell
+
+g43    (debug,g43 zoffset=#5403)
+g4p1   ;dwell
+
+g43h12 (debug,g43h12 zoffset=#5403)
+
+g0x0y0
+g1f1000 x1
+g1f1000 y1
+g0x0y0
+
+g49 (debug,g49 zoffset=#5403)
+t0
+m6
+m2
diff --git a/configs/sim/axis/db_demo/db.py b/configs/sim/axis/db_demo/db.py
@@ -0,0 +1,147 @@
+#!/usr/bin/python
+
+# Demonstrate LinuxCNC interface for a database of tools
+
+#  Example command line testing:
+#  $ ./db_demo.py
+#  g               (g: get all tools)
+#  p t11 p12 d.3   (u: put t11 for received update)
+#  g               (g: note changes from p cmd)
+
+# Notes
+#  1) all writes to stdout are piped to host
+#     (use stderr for debug prints)
+#-----------------------------------------------------------
+
+import sys
+db_ran_savefile    = "/tmp/db_ran_file"
+db_nonran_savefile = "/tmp/db_nonran_file"
+
+# Default: support random_toolchanger
+# Use symbolic link with "nonran" in its name to
+# use nonrandom_toolchanger provisions
+if sys.argv[0].find("nonran")>=0:
+    sys.stderr.write("%s:starting(nonrandom_toolchanger)\n"%sys.argv[0])
+    random_toolchanger = 0
+    db_savefile = db_nonran_savefile
+else:
+    sys.stderr.write("%s:starting(random_toolchanger)\n"%sys.argv[0])
+    random_toolchanger = 1
+    db_savefile = db_ran_savefile
+
+#-----------------------------------------------------------
+
+from tooldb import tooldb_callbacks # functions
+from tooldb import tooldb_tools     # list of tool numbers
+from tooldb import tooldb_loop      # main loop
+
+#-----------------------------------------------------------
+# Simulate a database of tools, using a flat file:
+
+tools = dict() # global
+
+def save_tools_to_file(fname):
+    global tools
+    f = open(fname,"w")
+    for k in tools:
+        if debug: sys.stderr.write("Save: key=%s %s\n"%(k,tools[k]))
+        f.write(tools[k]+"\n")
+    f.close()
+
+def home_pocket(toolno):
+    # simulation rule:
+    return toolno+1 # make home pocket number differ from tool number
+
+def init_tools(tfirst,tlast):
+    # set tool parameters for all tools
+    global tools
+    for i in range(tfirst,tlast+1):
+        tools["T%d"%i] = "T%-3d P%-3d Z0.%d D0.%d"%(i,home_pocket(i),i,i)
+    if random_toolchanger:
+        tools["T0"] = "T0   P0 ; Empty pocket"
+    save_tools_to_file(db_savefile)
+    return
+
+def make_tools(tfirst,tlast):
+    global tools
+    try:   f = open(db_savefile,"r")
+    except IOError:
+        init_tools(tfirst,tlast)
+        return
+    except Exception as e:
+        sys.stderr.write("make_tools: e=%s\n"%e)
+
+    # use opened file:
+    while True:
+        line = f.readline().strip()
+        if not line: break
+        key = line[0:line.index(" P")].strip()
+        tools[key] = line
+        if debug: sys.stderr.write("key=%s line=%s\n"%(key,line))
+    f.close()
+    save_tools_to_file(db_savefile)
+    return
+
+#-----------------------------------------------------------
+# functions to simulate db queries/transactions
+
+#   user_get_tool: host requests tool data
+def user_get_tool(toolno):
+    # host requests tool data
+    key = "T%d"%toolno
+    if debug: sys.stderr.write("@@user_get_tool: %s\n"%tools[key])
+    return tools[key]
+
+#   user_put_tool_*: host updates tool data:
+#                    1) offset change (G10 L1, G10 L2 ...)
+#                    2) tool removed from spindle
+def user_put_tool_ran_tc(toolno,params):
+    key = "T%d"%toolno
+    tools[key] = params.upper() # sync to host
+    if debug: sys.stderr.write("@@user_put_tool_ran_tc: %s\n"%tools[key])
+    save_tools_to_file(db_savefile)
+    return
+
+lastp0=""
+def user_put_tool_nonran_tc(toolno,params):
+    global lastp0
+    key = "T%d"%toolno
+    t = params.split()[0].strip().upper()
+    p = params.split()[1].strip().upper()
+    if p == "P0":
+        if lastp0 != "":
+            # restore prior p0 entry
+            rkey = lastp0.split()[0].strip().upper()
+            tools[rkey] = lastp0 # restore prior p0 settings
+    if toolno == 0:
+        if debug: sys.stderr.write("Spindle unloaded (T0 P0)\n")
+        save_tools_to_file(db_savefile)
+        return
+    lastp0 = tools[key] # save p0 settings
+    tools[key] = params.upper() # sync to host
+    if debug: sys.stderr.write("@@user_put_tool_nonran_tc: %s\n"%tools[key])
+    save_tools_to_file(db_savefile)
+#-----------------------------------------------------------
+# begin
+debug = 0 # debug var for command line testing
+if (len(sys.argv) > 1 and sys.argv[1] == 'debug'): debug = 1
+
+toolno_min = 10
+toolno_max = 19
+make_tools(toolno_min,toolno_max)
+
+if random_toolchanger:
+    tooldb_callbacks(user_get_tool,user_put_tool_ran_tc)
+else:
+    tooldb_callbacks(user_get_tool,user_put_tool_nonran_tc)
+
+toollist = range(toolno_min,toolno_max+1)
+if random_toolchanger: toollist.append(0) # T0 provision
+tooldb_tools(toollist)
+
+try:
+    tooldb_loop()  # loop forever, use callbacks
+except Exception as e:
+    if sys.stdin.isatty():
+        print(("exception=",e))
+    else: pass # avoid messages at termination
diff --git a/configs/sim/axis/db_demo/db_editor.tcl b/configs/sim/axis/db_demo/db_editor.tcl
@@ -0,0 +1,8 @@
+#!/usr/bin/tclsh
+package require Tk
+wm withdraw .
+tk_messageBox \
+    -title "Placeholder" \
+    -type ok \
+    -message "Specify a\n\n\[DISPLAY\]TOOL_EDITOR\n\napplicable for the\n\n\[EMCIO\]DB_PROGRAM"
+exit 0
diff --git a/configs/sim/axis/db_demo/db_nonran.ini b/configs/sim/axis/db_demo/db_nonran.ini
@@ -0,0 +1,15 @@
+#INCLUDE base.inc
+
+[EMC]
+VERSION = 1.1
+MACHINE = db_nonran NONRANDOM toolchanger
+
+[RS274NGC]
+PARAMETER_FILE = db_nonran.var
+
+[EMCIO]
+EMCIO = io
+CYCLE_TIME = 0.1
+RANDOM_TOOLCHANGER   = 0
+DB_PROGRAM           = ./db_nonran.py
+# TOOL_TABLE= is not used with DB_PROGRAM
diff --git a/configs/sim/axis/db_demo/db_nonran.py b/configs/sim/axis/db_demo/db_nonran.py
@@ -0,0 +1 @@
+db.py
diff --git a/configs/sim/axis/db_demo/db_ran.ini b/configs/sim/axis/db_demo/db_ran.ini
@@ -0,0 +1,19 @@
+#INCLUDE base.inc
+
+[EMC]
+VERSION = 1.1
+MACHINE = db_ran RANDOM toolchanger
+
+[RS274NGC]
+PARAMETER_FILE = db_ran.var
+
+[EMCIO]
+EMCIO = io
+CYCLE_TIME = 0.1
+RANDOM_TOOLCHANGER   = 1
+DB_PROGRAM           = ./db_ran.py
+# TOOL_TABLE= is not required for DB_PROGRAM
+# For random toolchangers, a file named:
+#     db_spindle.tbl
+# is automatically created to track tool in spindle
+
diff --git a/configs/sim/axis/db_demo/db_ran.py b/configs/sim/axis/db_demo/db_ran.py
@@ -0,0 +1 @@
+db.py
diff --git a/configs/sim/gmoccapy/python/stdglue.py b/configs/sim/gmoccapy/python/stdglue.py
@@ -1,3 +1,8 @@
+#NOTE:
+#     The legacy names *selected_pocket* and *current_pocket* actually reference
+#     a sequential tooldata index for tool items loaded from a tool
+#     table ([EMCIO]TOOL_TABLE) or via a tooldata database ([EMCIO]DB_PROGRAM)
+
 #   This is a component of LinuxCNC
 #   Copyright 2014 Norbert Schechner <nieson@web.de>
 #

diff --git a/configs/sim/sim.tbl b/configs/sim/sim.tbl
@@ -1,4 +1,4 @@
 T1 P1 D0.125000 Z+0.511000 ;1/8 end mill
 T2 P2 D0.062500 Z+0.100000 ;1/16 end mill
 T3 P3 D0.201000 Z+1.273000 ;#7 tap drill
-T99999 P99999 Z+0.100000 ;big tool number
+T98876 P543 Z+0.100000 ;big tool number
diff --git a/docs/man/man1/iocontrol.1 b/docs/man/man1/iocontrol.1
@@ -13,7 +13,7 @@ iocontrol \- accepts NML I/O commands, interacts with HAL in userspace
 
 
 .SH DESCRIPTION
-These pins are created by the userspace IO controller, usually found in $LINUXCNC_HOME/bin/io
+These pins are created by the userspace IO controller, usually found in $LINUXCNC_HOME/bin/io and loaded by ini file setting: [EMCIO]EMCIO=io.
 .P
 The signals are turned on and off in userspace - if you have strict timing requirements or simply need more i/o, consider using the realtime synchronized i/o provided by \fBmotion\fR(9) instead.
 .P
@@ -81,11 +81,10 @@ mechanism) of the tool requested by the most recent T-word.
 \fBiocontrol.0.user\-request\-enable 
 (Bit, Out) TRUE when the user has requested that estop be cleared
 
-.SH PARAMETERS
 
 .TP
 \fBiocontrol.0.tool\-prep\-index
-(s32, RO) IO's internal array index of the prepped tool requested
+(Bit, Out) IO's internal array index of the prepped tool requested
 by the most recent T-word.  0 if no tool is prepped.  On Random
 toolchanger machines this is tool's pocket number (ie, the same as the
 tool\-prep\-pocket pin), on Non-random toolchanger machines this is